1. Field of the Invention
The present invention relates to a system and methodology for improving the Internet delivery speed and reliability of digital photographs from scan centers, where these images were originally scanned from photographic prints, to the user's browsers during high-volume periods of e-commerce transactions.
2. Description of the Background Art
Although digital cameras have become commonplace in consumer markets, most people still use cameras that capture images on celluloid film. Typically, the consumers physically take their exposed film to nearby drop-off centers for developing and printing. These customers later pick-up their processed photographs from wherever they initially had taken them. What the consumer gets for his or her two-stop effort is a single copy of the negatives and a single copy of typically 4″-by-6″ standard (positive) prints from those negatives.
Often this initial transaction is the finality of the commercial experience for processing a roll of film. However, people may want either additional or enhanced copies of their pictures. They may want individual photographs enlarged, or they may want a print (or enlargement) of a portion of a photograph (“cropping”), or they may even want a picture printed on a tee shirt or some medium other than the conventional photo-print paper. In many situations, the non-professional photographer may want to share, or provide, copies or enhanced copies with friends, relatives, or associates, including sending out e-mail announcements about the pictures. Friends, relatives, or associates will frequently initiate the request for copies of the pictures. Thus in many instances, the decision to order more photo processing occurs at a later point in time from the initial transaction.
As can be expected, there is a litany of scenarios for wanting to order and purchase versions of the original photographs following the typical purchase of an initial roll of standard 4-by-6 prints. These usually necessitate inconvenience for the non-professional photographer to the extent that they often do not result in the consumers bothering to purchase the desired extra copies. Simply put, the necessary activities of filing, viewing, protecting, distributing, enhancing, and ordering hinder further commercial transactions.
Consider, for instance, what someone has to do to reproduce special photographs. Many people accumulate many developed rolls of film over time, and tend to store them somewhere, such as in several shoeboxes kept in a closet or attic. To find a particular photograph for either reproducing, enlarging, enhancing, sharing, or putting onto a coffee cup or tee shirt, the person could spend considerable time looking among a hoard of pictures, finding the matching negative, physically taking the negative to a photo center, placing the order (cropping requests are usually not even doable at most photo drop-off locations), physically returning to pick-up the new prints, and, if the goal is to distribute the prints to others, packaging them up for postage, and physically mailing them, sometimes from a post office.
Digitizing (analog) photographs offers many of the benefits and conveniences of digitized documentation. A digital photo, at sufficiently high resolution, provides a reasonable facsimile of the fidelity of the original from film. A digital photograph can be created from a print by scanning the original film photograph. In response to customer demand, retail photo developers are increasingly offering this capability as a standard service.
Today, the job of more conveniently storing and retrieving photographs is accomplished by making digital copies of the pictures, and recording those copies onto media such as floppy disks or CD-ROMs. This approach is currently provided by a number of retail photo-developing operators, which spawn dozens of distributed scan centers that make digital copies of customers' photographs at the same time the film is being developed. This commercial model is modestly successful at best, because it requires the customers to physically go to the nearest scanning center. Whereas the numerous photo film drop-off locations are very ubiquitous and relatively convenient, the much smaller group of distributed scan centers are more distant from the average customer.
Of course digital content lends itself to being transmitted across the Internet. The next approach for facilitating print reordering has been for the scan centers to make the digital images available on the Internet, or “Web,” which would conveniently enable easy (shared) viewing, enhancing, and ordering activities for both the original customers and their friends, relatives, or associates. However, the scan centers face the same dilemma regarding the Internet distribution of digital images as did the individual.
Consider the difficulty of sharing one's photos on-line. A person could distribute his or her own digital pictures to friends and family by email; but given the current bandwidth constraints—even when employing today's “broadband” communication (e.g., DSL or cable modem)—this approach is not practical for high-resolution photographic-quality digital images. Given the excessive transmission times required to transmit full-size digital photos, it is much more convenient to simply use the postal service.
This problem is particularly acute for high-volume commercial distribution over the Internet of full-size digital photographs. The main problem with the transmission and display of photographs over the Internet is that the photographs are very large. The standard full-size scanned photograph is approximately 1+megabytes in size, while the average home has a modem connection to the Internet that will transfer 28,000 to 56,000 bits per second, if the Internet traffic is not too busy. Even if a single photograph is displayed, the transfer takes several minutes, which is an unacceptable user experience. Typically, users wish to review a roll of photographs at a time, which consists of 12, 24, or 36 photographs.
As a partial solution to this problem, a method was implemented for commercializing reprints by displaying highly-compressed images or “digital facsimiles” across the Internet. Whereas the full-size digital images could serve the same function as the negatives from film, that is, for the scan centers to reprint the photographs, compressed images (representations needing less data) can be used for speedy Internet distribution for previewing purposes. For instance, a standard full-size digital image is stored in a format of 1536-by-1024 pixels. Upon compressing that digital image to a “photo display size” (460-by-306 pixels), such an image becomes a reasonably Internet-distributable picture with acceptable fidelity for display on a computer monitor.
Apart from the issue of image transfer, digital images must also somehow be identified with a particular individual. Both purchasing and processing transactions involve processing photographs that are referenced by a roll of film. Therefore, each full-size image from a roll can be further compressed to a “thumbnail” size—96-by-64 pixels—to represent each photograph when an entire roll is displayed. A group of 12, 24, or even 36 thumbnails can quickly be transmitted across the Internet representing the roll of pictures. These thumbnail-size images, although of very low fidelity, provide sufficient pictorial information for recognizing a particular photograph from among a group of them on a computer monitor.
With these three digital image resolutions, or sizes, the scan centers can distribute a roll of thumbnails fairly quickly across the Internet on-demand, distribute a single photo display size photograph less quickly across the Internet on-demand, and make commercial prints on order from the full-size format which is stored on disk at the scan center. In any event, the full-size representation is not shipped across the Internet during any e-commerce transaction.
However, in some cases the full-size representations must be involved during the real-time user experience on the Web. Whereas the photo display sizes are suitable for on-line viewing to re-order products utilizing the original full-frame of a photograph, they are insufficient for supporting on-line enhancements. For example, a user may want to reprint only a single person who is part of a group of people in a picture, and do so from the convenience of his or her personal computer. Current photo web sites allow a user to zoom-in on a selected area within the photo display size being displayed, where the user selects a sub-portion of the original to be magnified on his or her computer monitor. Here, the user first isolates the area around that sub-portion, cropping out the remainder of the composition of the picture, and then magnifies that subset of the picture to preview it to determine if this would make a satisfactory print. Unfortunately today, this process is rather limited if it is derived from the medium resolution of photo display size images.
In particular, whereas enlarging photographs printed from film results in graininess, enlarging digitized images is even more dissatisfying. The enlargements magnify the pixels themselves more than the photo image they intend, a result known as “pixelization.” Pixelization renders very blocky images, with poor edge detail and poor shade continuity. If the user crops away two-thirds of the picture, and then magnifies the selected third, the pixelized lowered resolution image would be disappointing. This experience is similar to what happens when one views an impressionism painting from the normal distance versus when one walks up closer to it (which is the same manipulation as magnifying a photograph). At some point one no longer sees the “impression,” but rather the artistic technique that creates that impression: a pattern-less mosaic of spots with no distinguishing line or borders.
To address this problem, the photo web server needs to map the same portion of the picture that the user is considering to the original full-size image, compress that portion to a photo display size format for the user's display, and transmit it back to the user in real-time. This technique gives the user a better idea of what the enlargement would look like when printed, because all printing is done from the original full-size digitized representations. FIG. 1 is a block diagram illustrating the cropping enhancement process.
As shown in FIG. 1, this process includes a user's browser displaying a photo display size picture 110, a portion of the entire image that the user crops to enlarge 120, a corresponding full-size photograph at the photo web server 130, a photo display size image enlarged from the cropped area mapped within the full-size original picture 140, and a photo display size of the cropped area displaying in the user's browser 150.
Viewing the full-framed photo display size photograph in his or her browser 110, the user interacts with web software to specify a portion of the picture to crop down to 120. The full-framed photo display size in the user's browser corresponds to the original full-size image at the scan center 130. The photo web server runs software that maps the user-specified cropped area within the original full-size image at the scan center 130. The photo web server transforms the cropped area from within the full-size image to a photo display size image frame 140. The photo web server finally ships the photo display size cropped area across the Internet to be displayed in the user's browser 150. The real-time enhancement experience for the Internet user occurs more slowly than simply rendering a photo display size image in the user's browser 150.
Each photo web server manipulates all of the three formats of the photographs it initially had scanned from the photographic prints: full-size, photo display size, and thumbnail. To implement the previously-described real-time on-line photo services, a relatively small “claim server” is deployed at each scan center. Users use the appropriate URL (universal resource locator) to go to the enterprise's web site, where they enter a claim number they received when they initially picked up their film prints and negatives. The claim number serves to identify the roll of digitized images that the user wants to view on the Internet, where the “roll” of digital images corresponds to the grouping of pictures that originated on a customer's roll of film. The scan centers, of which there are about 60 currently in the United States, are completely distributed with no central source of control or connectivity: they each have their own corresponding “claim service” provided by their local claim server.
The current distribution infrastructure employs insufficient bandwidth. Each scan center has a relatively small bandwidth to the Internet, generally between 128K (ISDN) and a T1 line (that is shared rather than dedicated). FIG. 2 is a block diagram illustrating the bottlenecked infrastructure on which the scan centers rely to transfer digital pictures to web users on-demand. As shown in FIG. 2, the current claim service system includes a claim server 200 for each scan center, a moderately-low bandwidth connection to the Internet 202, a photo web site to manage e-commerce 204, and a web browser 206 which displays the images for the user.
The basic on-line operation proceeds as follows: Using his or her browser 206, the user requests his or her roll of images from the web site server 204, which forwards that request to the appropriate remote claim server 200 at the scan center where his or her roll of images is stored. The claim server 200 then requests its local transform server to compress the (claimed) roll of full-size images to corresponding thumbnails, and the claim server 200 returns these to the remote web site server 204. The remote web site server 204, in turn, passes them on to the user's browser 206. Whenever the user selects one of the thumbnails for fuller viewing, his or her request passes to the web site server 204, and on to the same claim server 200. The claim server 200, in turn, transforms the full-size image, this time to a photo display size, and returns that photo display size to the web site server 204, which parlays the photo display size to the user's browser 206.
This entire process occurs on-demand in real-time. Although the use of image compression enables photo e-commerce, five major deficiencies remain in the infrastructure of the scan centers preventing a good user experience. The scan centers usually have insufficient bandwidth to the Internet. The (small) claim server at a scan center is unable to handle (cyclical) spikes in demand. The scan centers cannot justify the cost for satisfactory services that would primarily be utilized for a few hours a day (with a high load). The scan centers do not provide high availability or reliability among the large number (60) of claim servers that do not have a system for fail-over. The scan centers have inadequate storage capability to maintain images for a sufficient period of time.
The average volume of traffic for each scan center does not justify high enough bandwidth for fluctuations in demand. Spikes in demand on the claim server are frequent because most users would access at common times, e.g., after work from 5 pm to 10 pm. FIG. 3 is a graph illustrating this phenomenon. The corresponding surge in bandwidth requirement for such periodic surges results in real-time high-volume traffic.
Seasonal spikes also exist, for example, during the Christmas holidays. When overloaded with requests, the connections between the claim servers and the web site servers are often dropped, or this might cause a claim server to crash. Whenever this happens, the requests fail and no photos can be retrieved. In any event, there is still a long delay on the first on-demand roll claim attempt, which diminishes the user experience even when photographs are successfully retrieved on the first try.
The current distribution infrastructure also utilizes existing resources poorly. Most of the day when demand is moderate to low, the claim server sits idle. Further, the claim server is often inaccessible for various reasons at random periods of the day. Attempts to add servers to each scan center have failed to provide sufficient service during periods of peak demand. Worse yet, with a distributed network of 60 independent services, there are 60 points of failure. Whenever one claim server goes down, none of the requests by any users for that node are satisfied. The service from that site must wait until that server is brought back up.
In order to provide a better level of service, various queue-based recovery schemes have been employed that try to retrieve the photographs when there are failures. One of the problems with these attempts to solve the problem is the ad hoc nature of the recovery solutions. There is no central point of recovery, and there is no fail-over. This stems, in part, from the hardware employed at the various scan centers. For instance, scan centers often employ relatively small disk storage systems, due to cost and to variance in the amounts of photographs each is required to store. Therefore, they can only handle storage for a relatively short period before they run out of disk space. Customers need a system that can process re-ordering for extended periods of time.
All told, the overall existing solution, having claim servers from which to retrieve photographs, is not scalable. No matter how many claim servers or storage disks are added, they simply cannot guarantee a high quality of service in a cost-effective manner. It is not efficient to purchase huge servers for each scan center to be able to service the periodic surge of user requests for only the short period of time (five hours), and then to remain relatively idle during the rest of the 24-hour day, nor is it cost effective to purchase and retain huge under-utilized disks for commercially adequate storage time. As a result of all of these shortcomings, a better solution is needed.